1,723 research outputs found
Renyi Differential Privacy
We propose a natural relaxation of differential privacy based on the Renyi
divergence. Closely related notions have appeared in several recent papers that
analyzed composition of differentially private mechanisms. We argue that the
useful analytical tool can be used as a privacy definition, compactly and
accurately representing guarantees on the tails of the privacy loss.
We demonstrate that the new definition shares many important properties with
the standard definition of differential privacy, while additionally allowing
tighter analysis of composite heterogeneous mechanisms
Privacy via the Johnson-Lindenstrauss Transform
Suppose that party A collects private information about its users, where each
user's data is represented as a bit vector. Suppose that party B has a
proprietary data mining algorithm that requires estimating the distance between
users, such as clustering or nearest neighbors. We ask if it is possible for
party A to publish some information about each user so that B can estimate the
distance between users without being able to infer any private bit of a user.
Our method involves projecting each user's representation into a random,
lower-dimensional space via a sparse Johnson-Lindenstrauss transform and then
adding Gaussian noise to each entry of the lower-dimensional representation. We
show that the method preserves differential privacy---where the more privacy is
desired, the larger the variance of the Gaussian noise. Further, we show how to
approximate the true distances between users via only the lower-dimensional,
perturbed data. Finally, we consider other perturbation methods such as
randomized response and draw comparisons to sketch-based methods. While the
goal of releasing user-specific data to third parties is more broad than
preserving distances, this work shows that distance computations with privacy
is an achievable goal.Comment: 24 page
Differential Privacy in the Industry: Challenges and Successes
The views are the author's, and do not represent the views of any employers, past or present
Privacy Amplification by Iteration
Many commonly used learning algorithms work by iteratively updating an
intermediate solution using one or a few data points in each iteration.
Analysis of differential privacy for such algorithms often involves ensuring
privacy of each step and then reasoning about the cumulative privacy cost of
the algorithm. This is enabled by composition theorems for differential privacy
that allow releasing of all the intermediate results. In this work, we
demonstrate that for contractive iterations, not releasing the intermediate
results strongly amplifies the privacy guarantees.
We describe several applications of this new analysis technique to solving
convex optimization problems via noisy stochastic gradient descent. For
example, we demonstrate that a relatively small number of non-private data
points from the same distribution can be used to close the gap between private
and non-private convex optimization. In addition, we demonstrate that we can
achieve guarantees similar to those obtainable using the
privacy-amplification-by-sampling technique in several natural settings where
that technique cannot be applied.Comment: Extended abstract appears in Foundations of Computer Science (FOCS)
201
- …